Lower bounds for finding stationary points II: first-order methods
نویسندگان
چکیده
منابع مشابه
Lower Bounds for Finding Stationary Points of Non-Convex, Smooth High-Dimensional Functions∗
We establish lower bounds on the complexity of finding -stationary points of smooth, non-convex, high-dimensional functions. For functions with Lipschitz continuous pth derivative, we show that all algorithms—even randomized algorithms observing arbitrarily high-order derivatives—have worst-case iteration count Ω( −(p+1)/p). Our results imply that the O( −2) convergence rate of gradient descent...
متن کاملIteration Bounds for Finding the -Stationary Points for Structured Nonconvex Optimization
In this paper we study proximal conditional-gradient (CG) and proximal gradient-projection type algorithms for a block-structured constrained nonconvex optimization model, which arises naturally from tensor data analysis. First, we introduce a new notion of -stationarity, which is suitable for the structured problem under consideration. We then propose two types of first-order algorithms for th...
متن کاملLower bounds for witness-finding algorithms
Can a procedure that decides whether a Boolean formula has a satisfying assignment help to find such an assignment? The naïve adaptive “search-todecision” reduction uses a linear number of (quite weak) queries. Is there a lower bound on the number of queries required for a nonadaptive search-to-decision reduction? We report on lower bounds for various classes of queries. Most interesting types ...
متن کاملLower Bounds for Higher-Order Convex Optimization
State-of-the-art methods in convex and non-convex optimization employ higher-order derivative information, either implicitly or explicitly. We explore the limitations of higher-order optimization and prove that even for convex optimization, a polynomial dependence on the approximation guarantee and higher-order smoothness parameters is necessary. As a special case, we show Nesterov’s accelerate...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Mathematical Programming
سال: 2019
ISSN: 0025-5610,1436-4646
DOI: 10.1007/s10107-019-01431-x